Discovering human activities from binary data in smart homes

With the rapid development in sensing technology, data mining, and machine learning fields for human health monitoring, it became possible to enable monitoring of personal motion and vital signs in a manner that minimizes the disruption of an individual’s daily routine and assist individuals with difficulties to live independently at home. A primary difficulty that researchers confront is acquiring an adequate amount of labeled data for model training and validation purposes. Therefore, activity discovery handles the problem that activity labels are not available using approaches based on sequence mining and clustering. In this paper, we introduce an unsupervised method for discovering activities from a network of motion detectors in a smart home setting. First, we present an intra-day clustering algorithm to find frequent sequential patterns within a day. As a second step, we present an inter-day clustering algorithm to find the common frequent patterns between days. Furthermore, we refine the patterns to have more compressed and defined cluster characterizations. Finally, we track the occurrences of various regular routines to monitor the functional health in an individual’s patterns and lifestyle. We evaluate our methods on two public data sets captured in real-life settings from two apartments during seven-month and three-month periods

PhD forum: extracting similar patterns of behavior with a network of binary sensors

The aging population is continuously growing and this results in increasing the demands for using technologies to help to manage the rapidly growing sector of the elderly population. To contribute in this effort, we propose a method that can find similar patterns of behavior for extended durations. Our method uses motion sensors as a privacy-aware alternative to cameras. We compute three initial parameters to extract similar patterns of behavior: (1) movement in spot; (2) movement between rooms; and (3) movement within rooms. The three parameters demonstrate good similarity indicators for finding patterns of behavior between each pair of days

Dempster-Shafer based multi-view occupancy maps

Presented is a method for calculating occupancy maps with a set of calibrated and synchronised cameras. In particular, Dempster-Shafer based fusion of the ground occupancies computed from each view is proposed. The method yields very accurate occupancy detection results and in terms of concentration of the occupancy evidence around ground truth person positions it outperforms the state-of-the- art probabilistic occupancy map method and fusion by summing

A best view selection in meetings through attention analysis using a multi-camera network

Human activity analysis is an essential task in ambient intelligence and computer vision. The main focus lies in the automatic analysis of ongoing activities from a multi-camera network. One possible application is meeting analysis which explores the dynamics in meetings using low-level data and inferring high-level activities. However, the detection of such activities is still very challenging due to the often corrupted or imprecise low-level data. In this paper, we present an approach to understand the dynamics in meetings using a multi-camera network, consisting of fixed ambient and portable close-up cameras. As a particular application we are aiming to find the most informative video stream, for example as a representative view for a remote participant. Our contribution is threefold: at first, we estimate the extrinsic parameters of the portable close-up cameras based on head positions. Secondly, we find common overlapping areas based on the consensus of people’s orientation. And thirdly, the most informative view for a remote participant is estimated using common overlapping areas. We evaluated our proposed approach and compared it to a motion estimation method. Experimental results show that we can reach an accuracy of 74% compared to manually selected views

Detection of visitors in elderly care using a low-resolution visual sensor network

Loneliness is a common condition associated with aging and comes with extreme health consequences including decline in physical and mental health, increased mortality and poor living conditions. Detecting and assisting lonely persons is therefore important-especially in the home environment. The current studies analyse the Activities of Daily Living (ADL) usually with the focus on persons living alone, e.g., to detect health deterioration. However, this type of data analysis relies on the assumption of a single person being analysed, and the ADL data analysis becomes less reliable without assessing socialization in seniors for health state assessment and intervention. In this paper, we propose a network of cheap low-resolution visual sensors for the detection of visitors. The visitor analysis starts by visual feature extraction based on foreground/background detection and morphological operations to track the motion patterns in each visual sensor. Then, we utilize the features of the visual sensors to build a Hidden Markov Model (HMM) for the actual detection. Finally, a rule-based classifier is used to compute the number and the duration of visits. We evaluate our framework on a real-life dataset of ten months. The results show a promising visit detection performance when compared to ground truth

Riemannian Manifold-Based Support Vector Machine for Human Activity Classification in Images

This paper addresses the issue of classification of human activities in still images. We propose a novel method where part-based features focusing on human and object interaction are utilized for activity representation, and classification is designed on manifolds by exploiting underlying Riemannian geometry. The main contributions of the paper include: (a) represent human activity by appearance features from image patches containing hands, and by structural features formed from the distances between the torso and patch centers; (b) formulate SVM kernel function based on the geodesics on Riemannian manifolds under the log-Euclidean metric; (c) apply multi-class SVM classifier on the manifold under the one-against-all strategy. Experiments were conducted on a dataset containing 2750 images in 7 classes of activities from 10 subjects. Results have shown good performance (average classification rate of 95.83%, false positive 0.71%, false negative 4.24%). Comparisons with three other related classifiers provide further support to the proposed method

ICMI 2012 chairs' welcome

Welcome to Santa Monica and to the 14th edition of the International Conference on Multimodal Interaction, ICMI 2012. ICMI is the premier international forum for multidisciplinary research on multimodal human-human and human-computer interaction, interfaces, and system development. We had a record number of submissions this year: 147 (74 long papers, 49 short papers, 5 special session papers and 19 demo papers). From these submissions, we accepted 15 papers for long oral presentation (20.3% acceptance rate), 10 papers for short oral presentation (20.4% acceptance rate) and 19 papers presented as posters. We have a total acceptance rate of 35.8% for all short and long papers. 12 of the 19 demo papers were accepted. All 5 special session papers were directly invited by the organizers and the papers were all accepted. In addition, the program includes three invited Keynote talks. One of the two novelties introduced at ICMI this year is the Multimodal Grand Challenges. Developing systems that can robustly understand human-human communication or respond to human input requires identifying the best algorithms and their failure modes. In fields such as computer vision, speech recognition, and computational linguistics, the availability of datasets and common tasks have led to great progress. This year, we accepted four challenge workshops: the Audio-Visual Emotion Challenge (AVEC), the Haptic Voice Recognition challenge, the D-META challenge and Brain-Computer Interface challenge. Stefanie Telex and Daniel Gatica-Perez are co-chairing the grand challenge this year. All four Grand Challenges will be presented on Monday, October 22nd, and a summary session will be happening on Wednesday, October 24th, afternoon during the main conference. The second novelty at ICMI this year is the Doctoral Consortium—a separate, one-day event to take place on Monday, October 22nd, co-chaired by Bilge Mutlu and Carlos Busso. The goal of the Doctoral Consortium is to provide Ph.D. students with an opportunity to present their work to a group of mentors and peers from a diverse set of academic and industrial backgrounds and institutions, to receive feedback on their doctoral research plan and progress, and to build a cohort of young researchers interested in designing multimodal interfaces. All accepted students receive a travel grant to attend the conference. From among 25 applications, 14 students were accepted for participation and to receive travel funding. The organizers thank the National Science Foundation (award IIS-1249319) and conference sponsors for financial support

Behavior analysis for aging-in-place using similarity heatmaps

The demand for healthcare services for an increasing population of older adults is faced with the shortage of skilled caregivers and a constant increase in healthcare costs. In addition, the strong preference of the elderly to live independently has been driving much research on "ambient-assisted living" (AAL) systems to support aging-in-place. In this paper, we propose to employ a low-resolution image sensor network for behavior analysis of a home occupant. A network of 10 low-resolution cameras (30x30 pixels) is installed in a service flat of an elderly, based on which the user's mobility tracks are extracted using a maximum likelihood tracker. We propose a novel measure to find similar patterns of behavior between each pair of days from the user's detected positions, based on heatmaps and Earth mover's distance (EMD). Then, we use an exemplar-based approach to identify sleeping, eating, and sitting activities, and walking patterns of the elderly user for two weeks of real-life recordings. The proposed system achieves an overall accuracy of about 94%